SG-WRAM Schema Guided Wrapper Maintenance

The World Wide Web has become one of the most important connections to various sources of information. A large proportion of the data is embedded in HTML documents. This language serves the visual presentation of data in Internet browser, but does not provide semantic information for the data presented. This form of data presentation is, therefore, inappropriate for the demands of automated, computer assisted information management system. In particular, if data from different sources needs to be combined, it is necessary to develop special and often complex programs to automate the data extraction. Wrappers are specialized program routines to fulfil such tasks. They automatically extract data from Internet web sites and convert the information into a structured format. As the manual coding of wrappers is timeconsuming and error-prone process, different methods [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12] have been proposed to automate the wrapper generation process. As a rule, however, a specially developed wrapper is required for each individual data source, because of the different and unique structures of web sites. The WWW is also extremely dynamic and continually evolving, which results in frequent changes in the structures of web documents. Consequently, wrappers may stop working when the structures of the corresponding documents are changed no matter how they have been generated. It is often necessary to constantly update or even completely rewrite existing wrappers, in order to maintain the desired data extraction capabilities. The simplest way to maintain wrappers is to re-create wrappers using the new HTML documents. Obviously, this method is inefficient in that the maintenance depends mostly on the system developers. In this demo, we propose a novel schema-guided approach for wrapper maintenance, called SG-WRAM, which is based on our previous work, a schema-guided wrapper generation system (SG-WRAP[8,9]). SG-WRAP can generate a wrapper to extract data from an HTML document to produce an XML document conforming to the user-defined Schema. Although changes of HTML documents are extremely various, some features of desired information in previous document, e.g. syntactic features, data pattern, notation and underlying schemas are still preserved in the changed one. Syntactic features, data pattern and notation can be easily obtained from schemas, previous rules and extracting results. Therefore, it is feasible to recognize data items in the changed document using these features. Based on these observations, we fulfill the maintenance following four sequential steps. At First, syntactic features, data pattern and notation are obtained from the schema, previous rule and extracted results, then they are used to recognize the data items. After that, they are grouped according to the given schema. Each group is an instance of the given schema. At last, the representative instances are selected to re-induce the extraction rule. We name these four steps as features discovery, item recovery, block configuration and wrapper reparation respectively. Our schema guided method for wrapper maintenance has several unique features comparing to the related work. We make good use of schema, which is given by user during the process of wrapper generation, to assist the procedures of item recovery and block configuration; Our experience with real-life web documents shows that our method can deal with the changes from simple to complex including context shift, structural shift [12] and hybrid changes; In our system, we give different method for simple changes in which condition a part of the rule is disabled and the complex changes in which condition most of the rule is disabled. That makes the re-inducted rule more accurate and complete.